Building scalable digital library ingestion pipelines using microservices

CORE, a harvesting service offering access to millions of open access research papers from around the world, has shifted its harvesting process from following a monolithic approach to the adoption of a microservices infrastructure. In this paper, we explain how we rearranged and re-scheduled our old ingestion pipeline, present CORE's move to managing microservices and outline the tools we use in a new and optimised ingestion system. In addition, we discuss the ineffciencies of our old harvesting process, the advantages, and challenges of our new ingestion system and our future plans. We conclude that via the adoption of microservices architecture we managed to achieve a scalable and distributed system that would assist with CORE's future performance and evolution

Developing Infrastructure to Support Closer Collaboration of Aggregators with Open Repositories

The amount of open access content stored in repositories has increased dramatically, which has created new technical and organisational challenges for bringing this content together. The COnnecting REpositories (CORE) project has been dealing with these challenges by aggregating and enriching content from hundreds of open access repositories, increasing the discoverability and reusability of millions of open access manuscripts. As repository managers and library directors often wish to know the details of the content harvested from their repositories and keep a certain level of control over it, CORE is now facing the challenge of how to enable content providers to manage their content in the aggregation and control the harvesting process. In order to improve the quality and transparency of the aggregation process and create a two-way collaboration between the CORE project and the content providers, we propose the CORE Dashboard

Towards effective research recommender systems for repositories

In this paper, we argue why and how the integration of recommender systems for research can enhance the functionality and user experience in repositories. We present the latest technical innovations in the CORE Recommender, which provides research article recommendations across the global network of repositories and journals. The CORE Recommender has been recently redeveloped and released into production in the CORE system and has also been deployed in several third-party repositories. We explain the design choices of this unique system and the evaluation processes we have in place to continue raising the quality of the provided recommendations. By drawing on our experience, we discuss the main challenges in offering a state-of-the-art recommender solution for repositories. We highlight two of the key limitations of the current repository infrastructure with respect to developing research recommender systems: 1) the lack of a standardised protocol and capabilities for exposing anonymised user-interaction logs, which represent critically important input data for recommender systems based on collaborative filtering and 2) the lack of a voluntary global sign-on capability in repositories, which would enable the creation of personalised recommendation and notification solutions based on past user interactions

Predicting article quality scores with machine learning: The UK Research Excellence Framework

National research evaluation initiatives and incentive schemes have previously chosen between simplistic quantitative indicators and time-consuming peer review, sometimes supported by bibliometrics. Here we assess whether artificial intelligence (AI) could provide a third alternative, estimating article quality using more multiple bibliometric and metadata inputs. We investigated this using provisional three-level REF2021 peer review scores for 84,966 articles submitted to the UK Research Excellence Framework 2021, matching a Scopus record 2014-18 and with a substantial abstract. We found that accuracy is highest in the medical and physical sciences Units of Assessment (UoAs) and economics, reaching 42% above the baseline (72% overall) in the best case. This is based on 1000 bibliometric inputs and half of the articles used for training in each UoA. Prediction accuracies above the baseline for the social science, mathematics, engineering, arts, and humanities UoAs were much lower or close to zero. The Random Forest Classifier (standard or ordinal) and Extreme Gradient Boosting Classifier algorithms performed best from the 32 tested. Accuracy was lower if UoAs were merged or replaced by Scopus broad categories. We increased accuracy with an active learning strategy and by selecting articles with higher prediction probabilities, as estimated by the algorithms, but this substantially reduced the number of scores predicted

Aggregating The World's Open Access Research Papers

CORE offers seamless, unrestricted access to millions of research papers from around the world to everyone for free. CORE believes that knowledge can be used for the public good, and the more widely this knowledge can be distributed the better, for everyone

CORE

Le présent document présente le CORE, un service de recherche largement utilisé, qui donne accès aux publications de recherche en libre accès, provenant d’un réseau mondial de dépôts. CORE a été créé dans le but de permettre l’exploration de textes et de données de la littérature scientifique et donc de soutenir la découverte scientifique, mais il est maintenant utilisé dans un large éventail de cas

A Case Report: Building communities with training and resources for Open Science trainers

To foster responsible research and innovation, research communities, institutions, and funders are shifting their practices and requirements towards Open Science. Open Science skills are becoming increasingly essential for researchers. Indeed general awareness of Open Science has grown among EU researchers, but the practical adoption can be further improved. Recognizing a gap between the needed and the provided training offer, the FOSTER project offers practical guidance and training to help researchers learn how to open up their research within a particular domain or research environment. Aiming for a sustainable approach, FOSTER focused on strengthening the Open Science training capacity by establishing and supporting a community of trainers. The creation of an Open Science training handbook was a first step towards bringing together trainers to share their experiences and to create an open and living knowledge resource. A subsequent series of train-the-trainer bootcamps helped trainers to find inspiration, improve their skills and to intensify exchange within a peer group. Four trainers, who attended one of the bootcamps, contributed a case study on their experiences and how they rolled out Open Science training within their own institutions. On its platform the project provides a range of online courses and resources to learn about key Open Science topics. FOSTER awards users gamification badges when completing courses in order to provide incentives and rewards, and to spur them on to even greater achievements in learning. The paper at hand describes FOSTER Plus’ training strategies, shares the lessons learnt and provides guidance on how to reuse the project’s materials and training approaches

Death and the Societies of Late Antiquity

Ce volume bilingue, comprenant un ensemble de 28 contributions disponibles en français et en anglais (dans leur version longue ou abrégée), propose d’établir un état des lieux des réflexions, recherches et études conduites sur le fait funéraire à l’époque tardo-antique au sein des provinces de l’Empire romain et sur leurs régions limitrophes, afin d’ouvrir de nouvelles perspectives sur ses évolutions possibles. Au cours des trois dernières décennies, les transformations considérables des méthodologies déployées sur le terrain et en laboratoire ont permis un renouveau des questionnements sur les populations et les pratiques funéraires de l’Antiquité tardive, période marquée par de multiples changements politiques, sociaux, démographiques et culturels. L’apparition de ce qui a été initialement désigné comme une « Anthropologie de terrain », qui fut le début de la démarche archéothanatologique, puis le récent développement d’approches collaboratives entre des domaines scientifiques divers (archéothanatologie, biochimie et géochimie, génétique, histoire, épigraphie par exemple) ont été décisives pour le renouvellement des problématiques d’étude : révision d’anciens concepts comme apparition d’axes d’analyse inédits. Les recherches rassemblées dans cet ouvrage sont articulées autour de quatre grands thèmes : l’évolution des pratiques funéraires dans le temps, l’identité sociale dans la mort, les ensembles funéraires en transformation (organisation et topographie) et les territoires de l’empire (du cœur aux marges). Ces études proposent un réexamen et une révision des données, tant anthropologiques qu’archéologiques ou historiques sur l’Antiquité tardive, et révèlent, à cet égard, une mosaïque de paysages politiques, sociaux et culturels singulièrement riches et complexes. Elles accroissent nos connaissances sur le traitement des défunts, l’emplacement des aires funéraires ou encore la structure des sépultures, en révélant une diversité de pratiques, et permettent au final de relancer la réflexion sur la manière dont les sociétés tardo-antiques envisagent la mort et sur les éléments permettant d’identifier et de définir la diversité des groupes qui les composent. Elles démontrent ce faisant que nous pouvons véritablement appréhender les structures culturelles et sociales des communautés anciennes et leurs potentielles transformations, à partir de l’étude des pratiques funéraires.This bilingual volume proposes to draw up an assessment of the recent research conducted on funerary behavior during Late Antiquity in the provinces of the Roman Empire and on their borders, in order to open new perspectives on its possible developments. The considerable transformations of the methodologies have raised the need for a renewal of the questions on the funerary practices during Late Antiquity, a period marked by multiple political, social, demographic and cultural changes. The emergence field anthropology, which was the beginning of archaeothanatology, and then the recent development of collaborative approaches between various scientific fields (archaeothanatology, biochemistry and geochemistry, genetics, history, epigraphy, for example), have been decisive. The research collected in this book is structured around four main themes: Evolution of funerary practices over time; Social identity through death; Changing burial grounds (organisation and topography); Territories of the Empire (from the heart to the margins). These studies propose a review and a revision of the data, both anthropological and archaeological or historical on Late Antiquity, and reveal a mosaic of political, social, and cultural landscapes singularly rich and complex. In doing so, they demonstrate that we can truly understand the cultural and social structures of ancient communities and their potential transformations, based on the study of funerary practices

CORE-GPT: Combining Open Access Research and Large Language Models for Credible, Trustworthy Question Answering

In this paper, we present CORE-GPT, a novel question-answering platform that combines GPT-based language models and more than 32 million full-text open access scientific articles from CORE (https://core.ac.uk). We first demonstrate that GPT3.5 and GPT4 cannot be relied upon to provide references or citations for generated text. We then introduce CORE-GPT which delivers evidence-based answers to questions, along with citations and links to the cited papers, greatly increasing the trustworthiness of the answers and reducing the risk of hallucinations. CORE-GPT’s performance was evaluated on a dataset of 100 questions covering the top 20 scientific domains in CORE, resulting in 100 answers and links to 500 relevant articles. The quality of the provided answers and relevance of the links were assessed by two annotators. Our results demonstrate that CORE-GPT can produce comprehensive and trustworthy answers across the majority of scientific domains, complete with links to genuine, relevant scientific articles